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ABSTRACT: 

//; this paper we wilt illustrate the conditional probability on random t>mph. We will compute steady state 
which can defined as limits of stationary matrix S . The steady state will be computed for some examples. 
After that we will illustrate algorithm of random walk in weighted and tin weighted graph. 



Definitions: 

Markov chain, Markov model: Is a mathematical model that makes it possible to study complex systems by 
establishing a state of the system and then effecting a transition to a new state, such a transition being dependent 
only on the values of the current state, and not dependent on the previous history of the system up to that point 
[1]. 



Probability matrix P: Conditional probability concerning a discrete Markov chain giving the probabilities 
of change from one state to another [4]. 

A stationary probability vector S: Is defined as a vector that does not change under application of the 
transition matrix; that is, it is defined as a left eigenvector of the probability matrix, associated with eigenvalue 
1:SP=S[2]. 

Random Graph: Is a graph in which properties such as the number of graph vertices, graph edges, and 
connections between them are determined in some random way [3]. 

Main Results: 

Algorithm which calculate the steady state probability of random graph: 

Let P is a probability matrix (For which each raw sums to 1) , an let S is stationary matrix, then the distribution 
over states can be written as general with relation : S n+1 = S n P. 
Where ( n ) describe time period of the system. 

Then if at time n the system is in state 2 , then 3 times periods later at time n+3 the distribution is S n+3 = S n+2 p 
= (S n+I P ) P = S n+l P 2 and so on. 

The Algorithm: 

[1] Create associated probability matrix for a given graph. 
[2] Get stationary matrix S. 

[3] Compute S° P = S 1 which describe the initial distribution of graph. 

[4] Compute S n+1 P = S" until you get steady state probability which defined as : 

a. y = lun*_ WM ,CS)*. 

Example 1: The graph shown in Fig.(l) represent the probability connects vertices with each other. According to figure , 
vertex A has probability of 0.9 to connect with A and 0.075 to connect with B , 0.25 to connect with C. 
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Fig.(l) 

We will compute the steady state for this distribution as follows: 

r 0.9 0.075 0.0251 

[ 1 ] Create associated probability matrix P = 0.15 0.3 0.05. 

W.25 0.25 0.5 J 

[2] Consider the stationary matrix S = (0 1 &). 

[3] Calculate : 

[ 0.9 0.075 0.025] 
S°P=(0 1 0] 0.15 0.3 0.05 = 

Lo.25 0.25 0.5 J 
(0.15 0.S 0.05) 

[0.9 0.075 0.025] 
[4] S^CO-15 0.3 0.05} 0.15 0.8 0.05 =(0.2675 0.6637 0.0637} 

Lo.25 o.25 as J 
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S 3 P= (0.3575 


0.5682 0.0742} 0.15 
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Lo.25 


0.25 
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(04255 0.4999 0.0744} 
Compute steady state distribution as follows: 
y P = y I = y , then 
y(P-I) = 
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0.25 
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:(0 0} 



Create a system of equation then solve it: 
yl + 0.15 y2 + 0.025 y3 = 
0.075 yl - 0.2 y2 -I- 0.025 y3 = 
0.025/1 + 0.5/2- 0.5 y3 = 
And yl + Y2+ y3 = 1 
This imply that: 

(yl y2 y3}= (0.625 0.3125 0.0625 } 

This mean that the steady state probabilities indicated that 62.5% of vertices connected with A , 31.5% connect 
with B and 6.25% connect with C . 
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Example 2:A Simple Markov Weather Model: 

The probabilities of weather conditions, given the weather on the preceding day, can be represented by a 

. ... t ■ D [0.9 0.1] 
transition matrix: P = ^ ^ ^ 

The matrix P represents the weather model in which a sunny day is 90% likely to be followed by another sunny 
day, and a rainy day is 50% likely to be followed by another rainy day. The columns can be labelled "sunny" 
and "rainy" respectively, and the rows can be labelled in the same order.(P), , is the probability that, if a given 
day is of type i, it will be followed by a day of type j. Notice that the rows of P sum to 1 : this is because P is a 
stochastic matrix. 

Predicting the weather 

The weather on day is known to be sunny. This is represented by a vector in which the "sunny" entry 
is 100%, and the "rainy" entry is 0%: S° = (1 0] 



The weather on day 1 can be predicted by: 

s 1 = s°p= (i o)[J;;? 53 = (0.9 o.i). 



Thus, there is an 90% chance that day 1 will also be sunny. The weather on day 2 can be predicted in the same 
way: S 2 = S 1 P = S°P 2 = (0.9 0.1) [J^ =(0.86 0.14). 

And so on. 

Steady state of the weather 

In this example, predictions for the weather on more distant days are increasingly inaccurate and tend 
towards a steady state vector. This vector represents the probabilities of sunny and rainy weather on all days, 
and is independent of the initial weather. 

7 P = y I = y , then 

w «[« °&<° »»■ 

-0.1yl + 0.5y2 = 

yl + yl = 1 by solving these equations we have: 

(yl yZ) = ( 0.833 0.167) . In conclusion, in the long term, 83% of days are sunny. 

Example 3: Consider a company has two brands A,B .10 % market share using brand A and 40% use the other 
6 i-l If S = (0.1 0.4). 

5 1 = S°P = (0.1 0.4) [J^ (0.62 0.38). 

5 2 = (0.62 0.38) [J 1 J J^]= (0.724 0.276). 

5 3 = (0.724 0.276) [Jg = (0.744 0.255). 

And so on .then the steady state will be: (yl y2) ( [J® jJ ,2 J - [J J] ) = (0 0] . 
-0.2 yl + 0.6 y2 = 

yl + y2 = 1 by solving these equations we have: 

(yl y2] = ( 0.75 0.25) .this means that 75% use brand A and 25% use brand B. 
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Algorithm which calculate random graph: 

We will illustrate the algorithm for two kinds of graph , weighted and un weighted graph. 
For un weighted graph: 

Input: An undirected graph G( V,E ) with IVI > 1 , El > 0. 
Steps: 

[1] Look for random pair of nodes x,y that are nonadjacent . 
[2] If no such pair of nodes x , y exists , then output G . 
Else 

[3] Add element { x , y } to set E'. 
[4] Return to step 1 . 

End while. 
[5] Output G' . 

End algorithm. 

We find that output graph G is graph with the same number of connected component as G , and has 
maximum possible density as a clique. 



Example l:By using the previous algorithm find the output random graph shown in Fig. (4) 




Fig.(4) 



Input: Undirected graph with 10 vertices , and has 3 component , IEI = 8 edges. 
Steps: 

[1] Start randomly with two nonadjacent nodes {8 , 10} 

[2] Add element {8, 10} to set E. 

[3] Return to step 1 with another pair of nodes. 

[4] Output G . 

End algorithm. 

The result graph G shown in Fig.(5). 

© 




We find that the output graph is regular and complete. 

Lemma 1: The output graph generating by algorithm of random graph will be regular and complete. 
Example 2:For another graph shown in Fig. (6) with 10 vertices but 1 component. 



BjEBjSjljB 
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Fig.(6) 



Input: Undirected graph with 10 vertices and 1 component. 
Steps: 

[1] Start randomly with nonadjacent vertices s.t {1,6}. 
[2] Add element { 1,6} to set E'. 
[3] Return to step 1. 
[4] Output G'. 

End algorithm. Output graph shown in Fig. (7). 




Fig.(7) 

2b. For weighted graph: 

Input: An undirected weighted graph G( V,E ) with IVI > 1 , IEI > 0. 
Steps: 

1 .Create the associated matrix . 

2. Normalize this matrix. 

3. Get the first transition probability matrix P . 

4. Expand by taking P th power of P. 

5. Get stationary matrix S. 

6. Compute S° P = S 1 which describe the initial distribution of graph. 

7. Compute S n+1 P = S n until you get steady state probability which defined as : 

j = lim^^CS} 71 . 

Notes: l.When normalize matrix in this algorithm , column sums to one. 

2.Each power of P consider as posibility distribution of random graph. 
Example 3:Calculate random graph for weighted graph shown in Fig.(8). 
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Fig.(8) 



ro 2 1 31 

1. Associated matrix is: 2 2 

b o o oJ 

r 1/2 1 3/5i 

2. Normalize the matrix : 1/3 2/5 

Ll/6 J 

rl9/30 3/10 1/5 1 

3. P 2 = P.P= 1/5 11/30 1/3 1/5 

L 1/12 1/6 1/loJ 



r 1/5 5/12 19/30 25/501 
P 3 = P 2 P= 5/1S 1/5 1/5 4/15 And so on. 
Lll/45 1/20 1/30 J 



/n\ ft) 1/2 1 3/5] n 01 /ol 

4.Get steady state : 1(1/3 2/5 - 1 ) = JJ 

W Li/6 oo o J b o i d w 



Note: We normalize the matrix by dividing the column(Which expressed a vertex V) by sums of weights of all 
edges coming out of this vertex. 

As example,vertex 1 adjacent with 3 edges of weights 2,3,1 then we divide columnl by (2+3+1) and so on. 
Example 4: For graph shown in Fig. (9) calculate two possibility distributions for random graph. 




Then yl = 



y3 = 
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rO 5/10 0-1 
5. Normalize matrix: 1 3/4 2 / 5 = P. 

Lo 3/10 1/5 oJ 



rS/10 15/40 10/50 l 

6. P 2 = P.P= 161/200 2/20 3/20 2/5 
La/10 2/50 11/40 6/50 1/sJ 



Application: Steady state for random bipartite graph: 

Definition of random bipartite graph: Is a bipartite graph which two sets can be randomly joined with each 

other , such that the first set can be considered as begin then the other set consider as begin. 

In random bipartite graph , we will compute steady state two times by changing the beginning set. 

Example 4:Consider a bipartite graph as shown in Fig. (2) which consist of two sets : { 1,2,3 } and { A,B,C}, 
compute the steady state for this graph when set { 1,2,3 } is beginning set , then when { A,B,C }is beginning set. 




Fig.(2) 
ri 01 

First , the incident matrix of this graph is: 1 2 1 2| 

rl/3 
Normalize the matrix: 2/3 1/4 
L 3/4 
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■1/3 1 , c 
2/3 1/4 2/6 =[ I/* 
« 3/4 4/ 6 J W 



D rl/3 ° W » \ 
S 2 = [V*] 2/3 1/4 2/6= 5/16 a 

WJ L 3/4 4/&I \U/J 
Then the steady state can obtained as follows: 

U ( 2/3 1/4 1/3 - 1 £> )= ») 
W L 3/4 2/3J 10 lJ W 



L 3/4 2/3J Lo 
-2/3 yl=0 yl = , 

2/3 rl - 3/4 y2 + 1/3 r3 =0 , yl + r? + y3 = 1 
By solving these equations we find that: 

We can compute steady state for the same graph by considered the set { 1,2,3} is the beginning set as follows: 



the incident matrix of this graph will be: 1 3 
Lo 2 4. 

[1 2/5 1 
Normalize the matrix: 1/5 3/ 7 , S°= I [ J 
lO 2/5 4/7J p 

rl 2/5 1,0, 
Then:S'= ? */5 3/7 1 = [*/*) 
l ^lo 2/5 4/7J W 
2/5 1 f2/5^ 

and so on. 

.0 2/5 4/7J V2/5/ 
Then the steady state can obtained as follows: 
2/5 1 
1/5 3/7 
2/5 4/7J 



len the steady state can obtained as 
fn , rl 2/5 1 n 0] 
H( « l/S 3/7 - 1 ) = (oJ 

w Lo 2/5 4/7J Lo 1J 

-4/5 3/7 = o 
W[ fl 2 / S _3 /7 J W 

After Creating a system of equation and solve it we find that: 

©-©■ 

by com pairing the two states we find that yl at first has highst steady state and lowest steady state when we 
consider set { 1,2,3 } is the beginning set , the same note for y2 , y3 . 

Example 2:for bipartite graph shown in Fig. (3) , compute the steady state for 
each vertex. 



BjEBjSjljB 
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[-2/3 3/8 1 f m 
2/3 -3/aJ w 



First , the incident matrix of this graph is : ^ 

[2/6 3/31 
Normalize the matrix :[ 4/6 &/ ^J 

Then the steady state can obtained as follows: 
fe) ( L/3 5/sl ~ [(> J 3 = 

since yl + y2 = 1 .. -2/3 yl + — = C then 
\y2/ V 16/23/ 

We can compute steady state for the same graph by considered the set { 1,2} is the beginning set a 
follows: 

the incident matrix of this graph will be: ^ ^] 

[2/5 4/91 
Normalize the matrix: [ g g p J 



: W [ 3/S -4/J"(cJ 



L 3/S -4/9J" 
3/5yl + 4/9y2 = 
yi + y2 = l 

(S) = (Ir/rs)" Which chan 8 e of the flrst state ' 
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